Salt Lake County COVID-19 Data

Since the beginning of the pandemic, Salt Lake County has been tracking case rates, positive test rates, and other COVID-19 related metrics. This data is available to the public at on the Utah state website. The data available via the county portal is divided into numerous subsets (.csv files), many of which are not relevant to our study. The data that will be used in our analysis are as follows.

• Testing_7day Rolling Average Percent Positivity Test-Test.csv
• Overview_Seven-Day Rolling Average COVID-19 Cases by Test Report Date.csv

The first of these data sets, Testing_7day Rolling Average Percent Positivity Test-Test.csv, provides the rolling 7-day average percent positive test rate for Salt Lake County by date, which we will use to determine if Zoom attendance at the University of Utah is correlated with this value.The second data set, Overview_Seven-Day Rolling Average COVID-19 Cases by Test Report Date.csv, provides several data points by date. Of those provided, we will be using the confirmed case count and 7-day average confirmed case count. We will use both of these data points to determine if there is a correlation between each of them and the Zoom attendance rate at the University of Utah.

University of Utah COVID-19 Data

The University of Utah is tracking COVID-19 data among its students and staff and making this data available on its website. The university is tracking positive cases and the 7-day average for positive cases and is posting it on their public site. We have written a web scraper to pull the data from their website each day so that we can record it, and we will be using this data to determine if there is a relationship between positive COVID-19 cases at the University of Utah and Zoom attendance among U of U students.

One limitation of the University of Utah COVID-19 data is that we are constrained by the University’s own sampling strategy for COVID-19 data. Without a clear understanding of how this data is gathered and the uncertainty present in the methodology used to produce these observations, our confidence in our results is necessarily reduced by an unquantifiable amount. Even if we find a correlation between reported University COVID-19 case numbers and Zoom lecture attendance, this inherent validity issue means we know, at best, of a correlation between reported numbers and attendance - not actual case numbers and attendance.

Weather Data

Temperature data for the day of each lecture will be obtained from OpenWeatherMap for the Salt Lake City area (City ID: 5780993). The temperature will be recorded two hours before the class, one hour before the class, and when the class starts, as well as at 6AM and 6PM. The weather conditions, as coded by OpenWeatherMap (cloudy, sunny, raining, etc.), will also be recorded two hours before the class starts.

In order to gather this data, we will use the web scraper we built to poll OpenWeatherMap. The scraper will collect Salt Lake City temperature, to the hundredth of a degree centigrade, two hours before, one hour before, and precisely when class starts (\(\pm 5\) minutes). As with any weather data, there is the possibility of measurement error introduced by weather observation equipment. However, the uncertainty introduced by possible measurement error is negligible relative to that introduced by the obstacles in our other datasets. Additionally, since temperatures are being manually copied and pasted between Discord and our data spreadsheet, there is a small, unquantifiable possibility of human error in transferring the data that may impact data reliability.

Zoom Attendance Data

Zoom attendance data will be collected directly from Zoom for hybrid Computer Science courses at the University of Utah. The courses sampled from will be,

• CS 3500
• CS 4400
• CS 3200
• CS 5140
• CS 3130

Data will be collected every lecture spanning roughly 10 weeks, from January 18, 2022 to March 24, 2022. Zoom attendance data will be collected three times per lecture: 10 minutes after the class starts, halfway through class, and 10 minutes before the end of class. Attendance then will then be calculated by taking the total number of participants in the Zoom lecture and subtracting one for the teacher and one for the researcher (us) collecting the data if they would not have otherwise attended the Zoom lecture.

Enrollment data for each course will be recorded near the end of the collection period, on March 25, 2022. This information will be acquired from University of Utah’s Campus Information System (CIS) to determine the number of students enrolled in each course at that point in time.

Temperature

gData <- read.csv("./Data/3130 Final Project Data Sheet - Mean Atd Rate, Mean Daily Temp, & COVID-19.csv")
gData$date <- as.Date(gData$date_2, format = "%m/%d/%Y")
 
accumulate_by <- function(dat, var) {
  var <- lazyeval::f_eval(var, dat)
  lvls <- plotly:::getLevels(var)
  dats <- lapply(seq_along(lvls), function(x) {
    cbind(dat[var %in% lvls[seq(1, x)], ], frame = lvls[[x]])
  })
  dplyr::bind_rows(dats)
}
 
gData <- gData %>% accumulate_by(~date)
 
pAvgTemp = ggplot(gData, aes(x=date, y=mean_daily_temp, frame=frame)) + geom_line() + labs(title="Mean Daily Temperature Between Jan 18, 2022 to Mar 24, 2022", x = "Date", y = "Temperature (F)") + geom_smooth(method = "lm")
 
ggplotly(pAvgTemp) %>%
  animation_opts(
    frame = 100,
    transition = 0,
    redraw = FALSE
  ) %>%
  animation_slider(
    currentvalue = list(
      prefix = "Date"
    )
  )

From the graph above, we can see that the average temperature increased as the semester went on. This makes sense as the semester began in the winter and progressed into spring. Interestingly, we can see two particularly cold days on February 3rd (21 degrees) and February 23 (20 degrees) and one abnormally warm day on March 3 (56.5 degrees).

Thus, we can conclude that temperatures were generally increasing as the semester progressed.

COVID-19 Case Rates

First, we will look at the average number of daily case counts.

pAvgTemp <- ggplot(gData, aes(x=date, y=daily_confirmed_case_count)) + geom_line() + labs(title="Mean Daily COVID-19 Case Counts Between Jan 18, 2022 to Mar 24, 2022", x = "Date", y = "Case Counts") + geom_smooth(method = "loess")
 
ggplotly(pAvgTemp)

Here, we can see that the number of daily COVID-19 cases dropped exponentially when comparing the beginning and end of the semester. This could be explained by increasing vaccination rates, people being more cautious, or people not caring enough to get tested. Regardless of the reason, we can clearly see a downwards trend in COVID-19 cases.

We can then look at the confirmed case count seven day average.

pAvgTemp <- ggplot(gData, aes(x=date, y=positive_test_rate_7d_avg)) + geom_line() + labs(title="Positive Test Rate 7 Day Average Between Jan 18, 2022 to Mar 24, 2022", x = "Date", y = "Positive Test Rate") + geom_smooth(method = "loess")
 
ggplotly(pAvgTemp)

The graph above shows a linear trend of decreasing positive test rates in Salt Lake County as the semester progresses.

Thus, these two graphs support the hypothesis that COVID-19 rates decreased over time.

Zoom Attendance Data

library(ggplot2)
library(plotly)
par(mar = c(4, 4, .1, .1))
 
attData <- read.csv("./Data/3130 Final Project Data Sheet - Atd & Weather.csv")
attData$date <- as.Date(attData$date, format = "%m/%d/%Y")
courseAttData <- split(attData, f=attData$course)
 
p3130 <- ggplot(courseAttData$`3130`, aes(x=date, y=mean_atd)) + geom_line() + geom_smooth(method = "lm")
 
p3200 <- ggplot(courseAttData$`3200`, aes(x=date, y=mean_atd)) + geom_line() + geom_smooth(method = "lm")
 
p3500 <- ggplot(courseAttData$`3500`, aes(x=date, y=mean_atd)) + geom_line() + geom_smooth(method = "lm")
 
p4400 <- ggplot(courseAttData$`4400`, aes(x=date, y=mean_atd)) + geom_line() + geom_smooth(method = "lm")
 
p5140 <- ggplot(courseAttData$`5140`, aes(x=date, y=mean_atd)) + geom_line() + geom_smooth(method = "lm")
 
fig <- subplot(p3130, p3200, p3500, p4400, p5140, nrows=3, margin=0.07, titleY = TRUE) %>%
  layout(title = "Mean Zoom Attendance Between Jan 18, 2022 to Mar 24, 2022 for Courses")
ann <- list(list(x = 0.15 , y = 1.01, text = "CS 3130", showarrow = F, xref='paper', yref='paper'),
           list(x = 0.85 , y = 1.01, text = "CS 3200", showarrow = F, xref='paper', yref='paper'),
           list(x = 0.15 , y = 0.61, text = "CS 3500", showarrow = F, xref='paper', yref='paper'),
           list(x = 0.85 , y = 0.61, text = "CS 4400", showarrow = F, xref='paper', yref='paper'),
           list(x = 0.15 , y = 0.21, text = "CS 5140", showarrow = F, xref='paper', yref='paper')
           )
fig <- fig %>% layout(annotations = ann)
 
fig

From the plots above, we can see that Zoom attendance varies wildly between different courses with many peaks and troughs. However, the one exception is CS 3200 where we can see that the number of Zoom attendees generally decreased as the semester went on. Otherwise, the other courses displayed a general trend of a relatively constant number of Zoom people throughout the semester. One common attribute shared among all the graphs was that the number of attendance dropped near March 1st.

Thus, we can conclude that the Zoom attendance rates remained mostly relatively constant throughout the semester.

Aggregate

Now that the preliminary evaluation of the data is complete, we can begin calculating correlation coefficients and confidence intervals to accompany them. Because we collected data on a large number of variables we have chosen to conduct a preliminary analysis using a subset of the variables and an aggregated value for our daily attendance rate. The following data was used in this analysis.

• Zoom attendance data collected at the University of Utah
• Temperature at 6AM and 6PM collected from OpenWeatherMaps
• Daily confirmed case count from the Salt Lake County dataset
• 7-day average case count from the Salt Lake County dataset
• 7-day average positive test rate from the Salt Lake County dataset

The correlations being evaluated in this section of the analysis are,

• Mean Attendance Rate vs Mean Daily Temperature
• Mean Attendance Rate vs Daily Confirmed New Case Count in Salt Lake County
• Mean Attendance Rate vs 7-Day Average Confirmed New Case Count in Salt Lake County
• Mean Attendance Rate vs 7-Day Average Positive Test Rate in Salt Lake County

To determine an attendance rate for each day Zoom attendance was recorded, an attendance rate was calculated for each course by averaging the three Zoom lecture headcounts for each course and then dividing that value by that course’s current enrollment. The enrollment rates for each day (each day had several rates, one for each of the courses that met that day) were then averaged to produce a single attendance rate value for each day on record. This is the Mean Atd Rate.

Due to the complexity of calculating correlation coefficients for each of the courses based on temperature data collected for a specific time before each class began, the 6AM and 6PM temperatures were averaged to get an average daily temperature for the time period during which the classes were in session.

All COVID-19 data was pulled directly from the .csv files downloaded from the Utah State Government site and used without modification.

A Pearson correlation coefficient was calculated for each of the dataset pairs and a Fisher Transformation was performed on the result to yield a normally distributed variable.

A 95% confidence interval was then calculated for each of the correlation coefficients and the upper and lower bounds of the intervals was determined.

The following are the results of this analysis:

• Mean Attendance Rate vs Mean Daily Temperature
  • Correlation Coefficient: -0.0392
  • Confidence Interval: -0.3725, 0.3029
• Mean Attendance Rate vs Daily Confirmed New Case Count in Salt Lake County
  • Correlation Coefficient: -0.1932
  • Confidence Interval: -0.4987, 0.1551
• Mean Attendance Rate vs 7-Day Average Confirmed New Case Count in Salt Lake County
  • Correlation Coefficient: -0.1064
  • Confidence Interval: -0.4291, 0.2404
• Mean Attendance Rate vs 7-Day Average Positive Test Rate in Salt Lake County
  • Correlation Coefficient: -0.0472
  • Confidence Interval: -0.3793, 0.2957

Controlled for Course

In performing a more in-depth analysis with Pearson correlations, we must first define a function to calculate the Pearson coefficient of determination, or \(R^2\). This number, which will fall between 0 and 1, shows what percentage of the variation in the response (dependent) variable is explained by the explanatory (independent) variable. We can also redefine ?cor such that it becomes more readily usable with dplyr.

Since we are using dplyr for much of this analysis, we define the functions using lazyeval to support non-standard evaluation.

# define dyplyr-able function to calculate R^2
rsq <- function(.data, x, y) {
    rsq_(.data, lazyeval::lazy(x), lazyeval::lazy(y))
}
 
# define corollary function_ since we're using non-standard evaluation (NSE)
rsq_ <- function(.data, x, y) {
    require(lazyeval)
    summary(lm(lazy_eval(x, .data)~lazy_eval(y, .data)))$r.squared
}
 
# bonus:
# define dyplyr-able function to calculate Pearson correlation coefficient
cor <- function(.data, x, y) {
    cor_(.data, lazyeval::lazy(x), lazyeval::lazy(y))
}
 
# define corollary function_ since we're using non-standard evaluation (NSE)
cor_ <- function(.data, x, y) {
    stats::cor(x = lazyeval::lazy_eval(x, .data),
               y = lazyeval::lazy_eval(y, .data),
               use = "complete.obs")
}

With these functions defined, we ingest the raw data and appropriately transform, or “wrangle” it, such that we are able to visualize and analyze it.

# compute mean attendance and propagate NA
# compute attendance rate based on number enrolled
# convert course into a factor
# convert weather into a factor
# make date R-readable
# compute mean of temps 2 hours before class
data <- read.csv("data/atd_weather_full.csv") %>%
    rowwise() %>%
    mutate(mean_atd = mean(c(atd_start, atd_mid, atd_end))) %>%
    mutate(atd_rate = mean_atd / enrolled) %>%
    mutate(course = factor(course)) %>%
    mutate(weather = factor(weather)) %>%
    mutate(date = as.Date(date, "%m/%d/%Y")) %>%
    mutate(mean_temp = mean(temp_tm2, temp_tm1, temp_tm0, na.rm = TRUE))

Now that the data has been ingested, we pare down our explanatory variables to only those at the interval level of measurement. This includes: mean_temp, which represents the mean of temperature 2 hours before, 1 hour before, and at the time class begins; uu_new_cases, which represents the number of daily new COVID cases were recorded at the University of Utah; and uu_new_cases_7da, which represents a rolling, 7-day average of daily new COVID cases recorded at the University of Utah. We also include atd_rate, our response variable, which we compute by taking the mean of Zoom lecture attendees at each of our three data points, then dividing this number by the number of students enrolled in the course at the end of the data collection period.

Our goal here is to begin by focusing on only a handful of variables, which we hope will allow us to quickly see trends and correlations across many pairs of variables.

# filtering down to a choice few variables
data_filtered <- data %>%
    select(date, course, atd_rate, mean_temp,
           uu_new_cases, uu_new_cases_7da)

With our data fully transformed, we can begin to plot correlations. The quickest way to bootstrap this process is by making a scatter plot with ggplot2 and visually inspecting the initial results.

# begin by looking at mean temp
data_filtered %>%
    ggplot(aes(mean_temp, atd_rate, color = course)) +
    geom_point() +
    geom_smooth(method = "lm") +
    scale_color_manual(values = c("3130" = "purple",
                                  "3200" = "red",
                                  "5140" = "green",
                                  "3500" = "orange",
                                  "4400" = "blue"))

When we begin by looking at the correlation between mean_temp and atd_rate, shown above, we immediately see class-wise trends emerge. Even with bands of standard error, we see a fairly good fit for 4400, which trends slightly upward, and for 3500, which trends sideways. 3200, despite having wider error bounds, seems to mostly trend downward. Since we see different trends with different classes, it suggests class could be confounding variable we hadn’t earlier considered. This reality highlights the possibility that there may be unmeasured characteristics of the class itself that may interact with mean_temp and a student’s likelihood to attend class via Zoom. For example, it’s possible that for difficult or very important classes, students are relatively “inelastic”, or insensitive, to changes in weather, preferring to always attend in person. While we should nod to this possibility, we lack the data and instrumentation to fully capture this variable and decompose it into its atomic parts, which is a limitation of this study.

We can continue with ggplot2, now looking at uu_new_cases and uu_new_cases_7da as explanatory variables.

# Explanatory: `uu_new_cases`
data_filtered %>%
    ggplot(aes(uu_new_cases, atd_rate, color = course)) +
    geom_point() +
    geom_smooth(method = "lm") +
    scale_color_manual(values = c("3130" = "purple",
                                  "3200" = "red",
                                  "5140" = "green",
                                  "3500" = "orange",
                                  "4400" = "blue"))

# Explanatory: `uu_new_cases_7da`
data_filtered %>%
    ggplot(aes(uu_new_cases_7da, atd_rate, color = course)) +
    geom_point() +
    geom_smooth(method = "lm") +
    scale_color_manual(values = c("3130" = "purple",
                                  "3200" = "red",
                                  "5140" = "green",
                                  "3500" = "orange",
                                  "4400" = "blue"))

For uu_new_cases, the first thing we notice is that there are very few data points at the very high end of the x-axis. This is because, toward the beginning of our data collection period, there were 1 or 2 days of collection that occurred right at the end of a COVID surge. For 3200 and 4400, we see that the trend-lines do not even extend beyond around 50 cases. Since data collection for these classes began just a day or two after the surge had already dropped back down, no observations exist here at the higher end of our explanatory variable for these classes. Furthermore, with so few data at this high end, the error margins are too large for us to draw any preliminary conclusions from these trends. We might look at 5140, for which Zoom attendance rate appears to increase perhaps linearly or logarithmically with daily new cases. But then we could also look at 3500, for example, for which Zoom attendance rate almost certainly decreases with the daily case count.

If we look at the 7-day average, uu_new_cases_7da, we get slightly more interesting results. Again, margins of error here make our results from 5140 and 3130 not immediately useful to us. However, we can see trends emerging again from 3200, 4400, and 3500, for which Zoom attendance rate appears to increase, move sideways, and decrease with average cases, respectively.

With some initial trends in mind, we can begin computing correlation pairs for each class. This is a way to get slightly better depth of information, though we’re still in the exploratory stage of looking at the gathered data.

# This will be the general format for performing some
# exploratory analysis on variable pairs.
# We do this for each class.
p3500 <- data_filtered %>%
    filter(course == 3500) %>%
    select(!course) %>%
    ggpairs() +
    labs(title = "3500 Pairs")

Now that we have prepared our plots, we can plot them one-by-one.

With 3500, we see a few interesting correlations. We see uu_new_cases_7da has a significant negative correlation with atd_rate (\(p < 0.001\)), uu_new_cases with atd_rate (\(p < 0.01\)). Mean temperature seems to have no significance here.

With 3130, we see no interesting correlations of any significance. We see a correlation between uu_new_cases_7da and uu_new_cases, which we would expect for trivial reasons. We also see a correlation between case rates and date, which supports an assumption we alluded to earlier–that case rates have mostly decreased as the data collection period went on.

With 3200, we see some correlations of significance. uu_new_cases_7da and uu_new_cases appear to have strong positive correlations with atd_rate (\(p < 0.01\)), and mean_temp a slightly weaker, negative correlation with atd_rate (\(p < 0.01\)). We also see a correlation between atd_rate and date (\(p < 0.001\)), which, since case rates also correlate with date, makes it difficult to infer causality in the earlier correlations. What looks like COVID case rates predicting attendance might in fact be dropping Zoom attendance as the semester goes on, unrelated to COVID case numbers.

For 4400, we see a fairly strong positive correlation between mean_temp and atd_rate (\(p < 0.01\)) and between date and atd_rate (\(p < 0.01\)).

Finally, for 5140, we see essentially no correlations of interest and significance.

Takeaways:

• Zoom Attendance Rate vs. Mean Temp
  • CS 3500: (\(p > 0.05\)) – Not significant.
  • CS 3200: \(-0.564,\ R^2 = 0.318\) (\(p < 0.01\))
  • CS 4400: \(0.6,\ R^2 = 0.36\) (\(p < 0.01\))
• Zoom Attendance Rate vs. 7-day Avg Cases at University of Utah
  • CS 3500: \(-0.871,\ R^2 = 0.758\) (\(p < 0.001\))!!
  • CS 3200: \(0.818,\ R^2 = 0.669\) (\(p < 0.01\))
  • CS 4400: (\(p > 0.05\)) – Not significant.

Sampling Methodology

The Zoom attendance data for this study was collected using a convenience sample. Specifically, we chose to collect attendance data on the courses that we were currently enrolled in so as to make data collection easy. The implications of this are that our sample probably does a poor job of representing our target population’s Zoom attendance for hybrid Computer Science courses at the University of Utah. Therefore, it is difficult for us to confidently assert that our findings are fully representative of the population. A better approach would have been to randomly select classes from all of the possible hybrid Computer Science courses at the university (a simple random sample), which would have produced a sample whose characteristics are more similar to that of the population under study.

Zoom Attendance Counting Methodology

In our counting method for Zoom attendance, we took the total number of people on Zoom and subtracted one for the teacher and one for the researcher (if they would not have otherwize logged into the Zoom meeting). This approach neglected the fact that this count could potentially include non-students. For example, several of the classes under study had teaching assistants (TAs) that attended via Zoom and that we weren’t able to easily remove from the count. This may arbitrarily inflate attendance numbers when they attend and introduce increased variability in the attendance counts if the TAs attended inconsistently. We do not expect this to be a major issue as the proportion of TAs to students is low. One way to avoid this would have been to be more rigorous in our data collection by going through people’s names in Zoom to verify their student identity before including them in the count, thereby eliminating all TAs from the count.

Small Sample Size

Due to the short length of the university semester (15 weeks), we were unable to collect more than ~10 weeks worth of attendance data that includes 2-3 class periods (data points) per week for each class. Compounding the issue are days where data could not be collected like CS 3130 labs, midterms, spring break, and holidays. Ultimately, these reduced our total sample size making it more difficult for us to reliably discover correlations. To improve upon this issue, we could have increased the duration of our study to collect more data points; however, this would have required that we extend our study to other semesters, which would not have been feasible for the goals of this course.

Timing of COVID-19 Correlations

As part of our study, we made the assumption that COVID-19 cases impact attendance the day of the lecture. The original reasoning was that COVID-19 cases would reflect how many people were sick on a given day or that higher case rates would be concerning to students, both of which could cause them to attend lectures on Zoom. This is not a guarantee given the asymptomatic period of COVID-19 where someone may test positive but feel fine. In this scenario, they could still attend many classes in person before they find out, making our study weaker. Given this, a future experiment might benefit from examining COVID-19 case rates delayed from the date of the lecture, e.g., COVID-19 cases a few days before lecture.

Inaccurate COVID-19 Data

Our study assumes accurate COVID-19 data despite it not being guaranteed. The COVID-19 data we used for this study (from Salt Lake County, the state of Utah, and the University of Utah) may not accurately reflect COVID-19 case counts or positive test rates. This is likely due to multiple factors like limited tests and testing facilities and an inability to test the entire population regularly. Therefore, it is likely that the COVID-19 data we collected actually undercounts the actual case count by a some margin. This error, depending on how large it is, could be negligible given that we are simply trying to measure the general trend of COVID-19 cases in relation to Zoom attendance. However, given that there exists this uncertainty, we acknowledge that it introduces some level of uncertainty into our conclusions.

Weather Data

The temperature data that we collected was to-the-minute while the weather type data points (sunny, cloudy, etc.) were collected two hours before the associated class began. Because of this, any correlation that might be present between the type of weather and Zoom lecture attendance will have to be present in the data at the time minus two hours mark to be visible in our data set. If the correlation depends on the weather type at a different point in time before a lecture begins or on the trend of the weather type before lecture, our analysis will not show this. This simply means that we cannot be fully confident in any lack of correlation between the type of weather and Zoom lecture attendance.

Possible Confounding Variables

We have identified several confounding variables in our study that reduce our ability to make strong conclusions about any correlations found by our analysis. They are as follows.